Systems and Methods For Haptic Remote Control Gaming

ABSTRACT

Systems and methods for haptic remote control gaming are disclosed. In one embodiment a portable multifunction device receives information from a remotely controllable device. The portable multifunction device can be operable as a remote control for the remotely controllable device. The portable multifunction device may be a smartphone, a tablet computer, or another suitable electronic device. The portable multifunction device can determine a haptic effect based at least in part on the information received from the remotely controllable device. The portable multifunction device may generate a signal configured to cause an actuator to output the determined haptic effect. The portable multifunction device can output the signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/826,391, filed Mar. 14, 2013, entitled “Systems and Methods forHaptic Remote Control Gaming” which claims priority to U.S. ProvisionalPatent Application No. 61/678,908, filed Aug. 2, 2012, entitled “Methodand Apparatus for a Haptic Cat,” and claims priority to U.S. ProvisionalPatent Application No. 61/679,382, filed Aug. 3, 2012, entitled “Methodand Apparatus for a Haptic Cat,” the entirety of each of which is herebyincorporated by reference.

FIELD

The present disclosure relates generally to systems and methods forhaptic remote control gaming.

BACKGROUND

A remotely controllable device is typically connected with a remotecontrol through a wireless connection so that an operator can stand inone place and remotely control the operation of the remotelycontrollable device using the remote control. Some remotely controllabledevices include a camera and video data from the camera can betransmitted to the remote control through the wireless connection sothat the operator can view the video data on a display associated withthe remote control.

SUMMARY

Embodiments provide systems and methods for haptic remote control gamingsystems. For example, one disclosed method comprises receiving, by aportable multifunction device, information from a remotely controllabledevice, the portable multifunction device being operable as a remotecontrol for the remotely controllable device; determining, by theportable multifunction device, a haptic effect based at least in part onthe received information; generating, by the portable multifunctiondevice, a signal configured to cause an actuator to output the hapticeffect; and outputting, by the first portable multifunction device, thesignal. In another embodiment, a computer readable medium comprisesprogram code for causing a processor to perform such a method.

These illustrative embodiments are mentioned not to limit or define theinvention, but rather to provide examples to aid understanding thereof.Illustrative embodiments are discussed in the Detailed Description,which provides further description of the invention. Advantages offeredby various embodiments of this invention may be further understood byexamining this specification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and constitute apart of this specification, illustrate one or more examples ofembodiments and, together with the description of example embodiments,serve to explain the principles and implementations of the embodiments.

FIG. 1 illustrates a system depicting a remotely controllable device anda remote control in accordance with an embodiment;

FIG. 2 illustrates a remote control for haptic remote control gaming inaccordance with an embodiment;

FIG. 3 illustrates a remotely controllable device for haptic remotecontrol gaming in accordance with an illustrative embodiment;

FIG. 4 illustrates a system depicting illustrative devices for hapticremote control gaming in an illustrative computing environment inaccordance with an embodiment;

FIG. 5 illustrates a system depicting illustrative devices for hapticremote control gaming in an illustrative computing environment inaccordance with an embodiment;

FIG. 6 illustrates a flow chart directed to a method of generatinghaptic effects in accordance with an embodiment; and

FIG. 7 illustrates a flow chart directed to a method of generatinghaptic effects in accordance with an embodiment.

DETAILED DESCRIPTION

Example embodiments are described herein in the context of systems andmethods for haptic remote control gaming. Those of ordinary skill in theart will realize that the following description is illustrative only andis not intended to be in any way limiting. Other embodiments willreadily suggest themselves to such skilled persons having the benefit ofthis disclosure. Reference will now be made in detail to implementationsof example embodiments as illustrated in the accompanying drawings. Thesame reference indicators will be used throughout the drawings and thefollowing description to refer to the same or like items.

In the interest of clarity, not all of the routine features of theimplementations described herein are shown and described. It will, ofcourse, be appreciated that in the development of any such actualimplementation, numerous implementation-specific decisions must be madein order to achieve the developer's specific goals, such as compliancewith application- and business-related constraints, and that thesespecific goals will vary from one implementation to another and from onedeveloper to another.

Illustrative System for Haptic Remote Control Gaming

Referring now to FIG. 1, FIG. 1 shows an illustrative embodiment of asystem 100 for haptic remote control gaming. This illustrative systemincludes a remote control 110 and a remotely controllable device 120.The remote control 110 is configured to wirelessly control the remotelycontrollable device. In this embodiment, the remote control 110 is asmartphone that is running a remote control application and the remotelycontrollable device 120 is a remote-controlled car (or “RC” car). Thesmartphone displays a number of objects to allow the user to control theRC car, including a throttle, a simulated joystick to turn the car, anda brake pedal. To drive the RC car, the user can touch the throttle anddrag it forward or backward to speed up or slow down the RC car. Inaddition, the user can drag the joystick left or right to turn the RCcar, and can press the brake pedal to slow and stop the RC car.

In addition to being able to control the RC car, the smartphone can alsoreceive information from the RC car. For example, in this embodiment,the RC car has multiple collision sensors on it to sense when itcollides with an obstacle. It also has a sensor on its roof to sensewhen the car has rolled over and is no longer drivable. The RC car has aprocessor that receives signals from the sensors and then is able to usea transmitter also in the RC car to transmit the sensor information backto the smartphone. The smartphone receives the information from the RCcar and can output haptic effects to indicate collisions or roll-oversto the user. Thus, the user experiences a more immersive experience whendriving the car: the user can feel impacts with obstacles via hapticeffects, and can be notified of error conditions, such as when the carhas rolled over and needs to be flipped back onto its wheels.

This illustrative example is given to introduce the reader to thegeneral subject matter discussed herein. The disclosure is not limitedto this example. The following sections describe various additionalnon-limiting embodiments and examples of devices, systems, and methodsfor haptic remote control gaming.

Illustrative Remote Control

Referring now to FIG. 2, FIG. 2 illustrates a remote control 200 forhaptic remote control gaming in accordance with one embodiment. In theembodiment shown in FIG. 2, the remote control 200 comprises a housing205, a processor 210, a memory 220, a touch-sensitive display 230, ahaptic output device 240, a communication interface 250, and a speaker270. In addition, the remote control 200 is in communication with hapticoutput device 260, which may be optionally coupled to or incorporatedinto some embodiments. The processor 210 is in communication with thememory 220 and, in this embodiment, both the processor 210 and thememory 220 are disposed within the housing 205. The touch-sensitivedisplay 230, which comprises or is in communication with atouch-sensitive surface, is partially disposed within the housing 205such that at least a portion of the touch-sensitive display 230 isexposed to a user of the remote control 200. In some embodiments, thetouch-sensitive display 230 may not be disposed within the housing 205.For example, the remote control 200 may be connected to or otherwise incommunication with a touch-sensitive display 230 disposed within aseparate housing. In some embodiment, the housing 205 may comprise twohousings that may be slidably coupled to each other, pivotably coupledto each other or releasably coupled to each other.

In the embodiment shown in FIG. 2, the touch-sensitive display 230 is incommunication with the processor 210 and is configured to providesignals to the processor 210 or the memory 220 and to receive signalsfrom the processor 210 or memory 220. The memory 220 is configured tostore program code or data, or both, for use by the processor 210, whichis configured to execute program code stored in memory 220 and totransmit signals to and receive signals from the touch-sensitive display230. In the embodiment shown in FIG. 2, the processor 210 is also incommunication with the communication interface 250 and is configured toreceive signals from the communication interface 250 and to outputsignals to the communication interface 250 to communicate with othercomponents or devices such as one or more remotely controllable devices.In addition, the processor 210 is in communication with haptic outputdevice 240 and haptic output device 260. and is further configured tooutput signals to cause haptic output device 240 or haptic output device260, or both, to output one or more haptic effects. Furthermore, theprocessor 210 is in communication with speaker 270 and is configured tooutput signals to cause speaker 270 to output sounds. In variousembodiments, the remote control 200 may comprise or be in communicationwith fewer or additional components or devices. For example, other userinput devices such as a mouse or a keyboard, or both, may be comprisedwithin the remote control 200 or be in communication with the remotecontrol 200. As another example, remote control 200 may comprise and/orbe in communication with one or more accelerometers, gyroscopes, digitalcompasses, and/or other sensors. A detailed description of thecomponents of the remote control 200 shown in FIG. 2 and components thatmay be in association with the remote control 200 are described herein.

The remote control 200 can be any device that is capable of receivinguser input and communicating with a remotely controllable device. Forexample, the remote control 200 in FIG. 2 includes a touch-sensitivedisplay 230 that comprises a touch-sensitive surface. In someembodiments, a touch-sensitive surface may be overlaid on thetouch-sensitive display 230. In other embodiments, the remote control200 may comprise or be in communication with a display and a separatetouch-sensitive surface. In still other embodiments, the remote control200 may comprise or be in communication with a display and may compriseor be in communication with other user input devices, such as a mouse, akeyboard, buttons, knobs, slider controls, switches, wheels, rollers,joysticks, other manipulanda, or a combination thereof.

In some embodiments, one or more touch-sensitive surfaces may beincluded on or disposed within one or more sides of the remote control200. For example, in one embodiment, a touch-sensitive surface isdisposed within or comprises a rear surface of the remote control 200.In another embodiment, a first touch-sensitive surface is disposedwithin or comprises a rear surface of the remote control 200 and asecond touch-sensitive surface is disposed within or comprises a sidesurface of the remote control 200. In some embodiments, the remotecontrol device may comprise two or more housing components, such as in aclamshell arrangement or in a slideable arrangement. For example, oneembodiment comprises a remote control device having a clamshellconfiguration with a touch-sensitive display disposed in each of theportions of the clamshell. Furthermore, in embodiments where the remotecontrol 200 comprises at least one touch-sensitive surface on one ormore sides of the remote control 200 or in embodiments where the remotecontrol 200 is in communication with an external touch-sensitivesurface, the display 230 may or may not comprise a touch-sensitivesurface. In some embodiments, one or more touch-sensitive surfaces mayhave a flexible touch-sensitive surface. In other embodiments, one ormore touch-sensitive surfaces may be rigid. In various embodiments, theremote control 200 may comprise both flexible and rigid touch-sensitivesurfaces.

In various embodiments, the remote control 200 may comprise or be incommunication with fewer or additional components than the embodimentshown in FIG. 2. For example, in one embodiment, the remote control 200does not comprise a speaker 270. In another embodiment, the remotecontrol 200 does not comprise a touch-sensitive display 230, butcomprises a touch-sensitive surface and is in communication with adisplay. In other embodiments, the remote control 200 may not compriseor be in communication with a haptic output device 240, 260 at all.Thus, in various embodiments, the remote control 200 may comprise or bein communication with any number of components, such as in the variousembodiments disclosed herein as well as variations that would beapparent to one of skill in the art.

The housing 205 of the remote control 200 shown in FIG. 2 providesprotection for at least some of the components remote control 200. Forexample, the housing 205 may be a plastic casing that protects theprocessor 210 and memory 220 from foreign articles such as rain. In someembodiments, the housing 205 protects the components in the housing 205from damage if the remote control 200 is dropped by a user. The housing205 can be made of any suitable material including but not limited toplastics, rubbers, or metals. Various embodiments may comprise differenttypes of housings or a plurality of housings. For example, in someembodiments, the remote control 200 may be a portable device, handhelddevice, toy, gaming console, handheld video game system, gamepad, gamecontroller, desktop computer, portable multifunction device such as acell phone, smartphone, personal digital assistant (PDA), laptop, tabletcomputer, digital music player, etc. In other embodiments, the remotecontrol 200 may be embedded in another device such as a wrist watch,other jewelry, gloves, etc. Thus, in embodiments, the remote control 200is wearable.

In the embodiment shown in FIG. 2, the touch-sensitive display 230provides a mechanism to allow a user to interact with the remote control200. For example, the touch-sensitive display 230 detects the locationor pressure, or both, of a user's finger in response to a user hoveringover, touching, or pressing the touch-sensitive display 230 (all ofwhich may be referred to as a contact in this disclosure). In oneembodiment, a contact can occur through the use of a camera. Forexample, a camera may be used to track a viewer's eye movements as theuser views the content displayed on the display 230 of the remotecontrol 200, or the user's eye movements may be used to transmitcommands to the remotely controlled device, such as to change directionor to aim a weapon at a target. In this embodiment, haptic effects maybe triggered based at least in part on the viewer's eye movements. Forexample, a haptic effect may be output when a determination is made thatthe viewer is viewing content at a particular location of the display230. In some embodiments, the touch-sensitive display 230 may comprise,be connected with, or otherwise be in communication with one or moresensors that determine the location, pressure, a size of a contactpatch, or any of these, of one or more contacts on the touch-sensitivedisplay 230. For example, in one embodiment, the touch-sensitive display230 comprises or is in communication with a mutual capacitance system.In another embodiment, the touch-sensitive display 230 comprises or isin communication with an absolute capacitance system. In someembodiments, the touch-sensitive display 230 may comprise or be incommunication with a resistive panel, a capacitive panel, infrared LEDs,photodetectors, image sensors, optical cameras, or a combinationthereof. Thus, the touch-sensitive display 230 may incorporate anysuitable technology to determine a contact on a touch-sensitive surfacesuch as, for example, resistive, capacitive, infrared, optical, thermal,dispersive signal, or acoustic pulse technologies, or a combinationthereof.

In the embodiment shown in FIG. 2, haptic output devices 240 and 260 arein communication with the processor 210 and are configured to provideone or more haptic effects. For example, in one embodiment, when anactuation signal is provided to haptic output device 240, haptic outputdevice 260, or both, by the processor 210, the respective haptic outputdevice(s) 240, 260 outputs a haptic effect based on the actuationsignal. For example, in the embodiment shown in FIG. 2, the processor210 is configured to transmit a haptic output signal to haptic outputdevice 240 comprising an analog drive signal. However, the processor 210is configured to transmit a command to haptic output device 260, whereinthe command includes parameters to be used to generate an appropriatedrive signal to cause the haptic output device 260 to output the hapticeffect. In other embodiments, different signals and different signaltypes may be sent to each of one or more haptic output devices. Forexample, in some embodiments, a processor may transmit low-level drivesignals to drive a haptic output device to output a haptic effect. Sucha drive signal may be amplified by an amplifier or may be converted froma digital to an analog signal, or from an analog to a digital signalusing suitable processors or circuitry to accommodate the particularhaptic output device being driven.

A haptic output device, such as haptic output devices 240 or 260, can beany component or collection of components that is capable of outputtingone or more haptic effects. For example, a haptic output device can beone of various types including, but not limited to, an eccentricrotational mass (ERM) actuator, a linear resonant actuator (LRA), apiezoelectric actuator, a voice coil actuator, an electro-active polymer(EAP) actuator, a memory shape alloy, a pager, a DC motor, an AC motor,a moving magnet actuator, an E-core actuator, a smartgel, anelectrostatic actuator, an electrotactile actuator, a deformablesurface, an electrostatic friction (ESF) device, an ultrasonic friction(USF) device, or any other haptic output device or collection ofcomponents that perform the functions of a haptic output device or thatare capable of outputting a haptic effect. Multiple haptic outputdevices or different-sized haptic output devices may be used to providea range of vibrational frequencies, which may be actuated individuallyor simultaneously. Various embodiments may include a single or multiplehaptic output devices and may have the same type or a combination ofdifferent types of haptic output devices.

In various embodiments, one or more haptic effects may be produced inany number of ways or in a combination of ways. For example, in oneembodiment, one or more vibrations may be used to produce a hapticeffect, such as by rotating an eccentric mass or by linearly oscillatinga mass. In some such embodiments, the haptic effect may be configured toimpart a vibration to the entire remote control or to only one surfaceor a limited part of the remote control. In another embodiment, frictionbetween two or more components or friction between at least onecomponent and at least one contact may be used to produce a hapticeffect, such as by applying a brake to a moving component, such as toprovide resistance to movement of a component or to provide a torque. Inorder to generate vibration effects, many devices utilize some type ofactuator or haptic output device. Known haptic output devices used forthis purpose include an electromagnetic actuator such as an EccentricRotating Mass (“ERM”) in which an eccentric mass is moved by a motor, aLinear Resonant Actuator (“LRA”) in which a mass attached to a spring isdriven back and forth, or a “smart material” such as piezoelectric,electro-active polymers or shape memory alloys.

In other embodiments, deformation of one or more components can be usedto produce a haptic effect. For example, one or more haptic effects maybe output to change the shape of a surface or a coefficient of frictionof a surface. In an embodiment, one or more haptic effects are producedby creating electrostatic forces and/or ultrasonic forces that are usedto change friction on a surface. In other embodiments, an array oftransparent deforming elements may be used to produce a haptic effect,such as one or more areas comprising a smartgel. Haptic output devicesalso broadly include non-mechanical or non-vibratory devices such asthose that use electrostatic friction (ESF), ultrasonic surface friction(USF), or those that induce acoustic radiation pressure with anultrasonic haptic transducer, or those that use a haptic substrate and aflexible or deformable surface, or those that provide projected hapticoutput such as a puff of air using an air jet, and so on. U.S. patentapplication Ser. No. 13/092,484 describes ways that one or more hapticeffects can be produced and describes various haptic output devices. Theentirety of U.S. patent application Ser. No. 13/092,484, filed Apr. 22,2011, is hereby incorporated by reference.

In FIG. 2, the communication interface 250 is in communication with theprocessor 210 and provides wired or wireless communications, from theremote control 200 to other components or other devices. For example,the communication interface 250 may provide wireless communicationsbetween the remote control 200 and a remotely controllable device. Insome embodiments, the communication interface 250 may providecommunications to one or more other devices, such as another remotecontrol 200 and/or one or more remotely controllable devices, to allowusers to interact with each other at their respective devices. Thecommunication interface 250 can be any component or collection ofcomponents that enables the remote control 200 to communicate withanother component or device. For example, the communication interface250 may comprise a PCI communication adapter, a USB network adapter, oran Ethernet adapter. The communication interface 250 may communicateusing wireless Ethernet, including 802.11a, g, b, or n standards. In oneembodiment, the communication interface 250 can communicate using RadioFrequency (RF), Bluetooth, CDMA, TDMA, FDMA, GSM, WiFi, satellite, orother cellular or wireless technology. In other embodiments, thecommunication interface 250 may communicate through a wired connectionand may be in communication with one or more networks, such as Ethernet,token ring, USB, FireWire 1394, fiber optic, etc. In some embodiments,remote control 200 comprises a single communication interface 250. Inother embodiments, remote control 200 comprises two, three, four, ormore communication interfaces. Thus, in embodiments, remote control 200can communicate with one or more remotely controllable devices throughone or more communication interfaces and/or can communicate with one ormore other remote controls through one or more communication interfaces.

Illustrative Remotely Controllable Device

Referring now to FIG. 3, FIG. 3 illustrates a remotely controllabledevice 300 for haptic remote control gaming in accordance with oneembodiment. In the embodiment shown in FIG. 3, the remotely controllabledevice 300 comprises a housing 305, a processor 310, a battery 315, amemory 320, input and/or output devices 330, a haptic output device 340,and a communication interface 350. In addition, the remotely controlleddevice 300 is in communication with haptic output device 360, which maybe optionally coupled to or incorporated into some embodiments. Theprocessor 310 is in communication with the memory and, in thisembodiment, both the processor 310 and the memory 320 are disposedwithin the housing 305. The input and/or output device 330 shown in FIG.3 are disposed within the housing 305. In some embodiments, one or moreof input and/or output devices 330 may not be disposed within thehousing 305. For example, the remotely controllable device 300 may beconnected to or otherwise in communication with one or more input and/oroutput devices 330 disposed within a separate housing.

In the embodiment shown in FIG. 3, the battery 315 provides power tovarious components of the remotely controllable device 300 such asprocessor 310, memory 320, I/O device(s) 330, haptic output device 340,communication interface 350, haptic output device 360, or a combinationthereof. While one battery is shown in FIG. 3, one or more batteries maybe employed. Further, in some embodiments, battery 315 may be disposedwithin the housing or may be otherwise connected to theremotely-controllable device. In embodiments, status informationregarding one or more batteries of a remotely controllable device 300 issent to at least one remote control and the remote control(s) canprovide haptic, visual, and/or auditory cues to a user of a remotecontrol indicating a status of a battery in the remotely controllabledevice 300. For example, if a battery of a remotely controllable device300 has a remaining power level below a specified threshold, then aremote control 200 that is in communication with remotely controllabledevice 300 may output a haptic effect configured to indicate that theremotely controllable device 300 needs to be charged and/or that thebattery needs to be replaced. In another embodiment, a warning sound isplayed by the remote control 200 when the battery level of a remotelycontrollable device 300 is below a predetermined percentage level. Inone embodiment, a percentage of remaining battery life of a remotelycontrollable device 300 is displayed on a display of a remote control200. Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

In the embodiment shown in FIG. 3, the input and/or output device(s) 330are in communication with the processor 310 and are configured toprovide signals to the processor 310 or the memory 320. The memory 320stores program code or data, or both, for use by the processor 310 andthe processor 310 executes program code stored in memory 320 andreceives signals from the input and/or output devices 330. An inputand/or output device 330 can include any number of suitable devices suchas a camera, an accelerometer, a gyroscope, digital compass, microphone,weapon, wind speed sensor, depth sensor, speed sensor, velocity sensor,temperature sensor, force sensor, collusion detection sensor, and/orother sensors. In various embodiments, one or more sensors can includean optical sensor, a mechanical contact sensor, a magnetic sensor, apotentiometer, and/or any other suitable sensor.

In the embodiment shown in FIG. 3, the processor 310 is in communicationwith the communication interface 350 and is configured to receivesignals from the communication interface 350 and to output signals tothe communication interface 350 to communicate with other components ordevices. In addition, the processor 310 is in communication with hapticoutput device 340 and haptic output device 360 and is further configuredto output signals to cause haptic output device 340 or haptic outputdevice 360, or both, to output one or more haptic effects. In variousembodiments, the remotely controllable device 300 may comprise or be incommunication with fewer or additional components or devices. Forexample, other input devices such as a camera or a microphone, or both,may be comprised within the remotely controllable device 300 or be incommunication with the remotely controllable device 300. A detaileddescription of the components of the remotely controllable device 300shown in FIG. 3 and components that may be in association with theremotely controllable device 300 are described herein.

The remotely controllable device 300 can be any device that is capableof communicating with a remote control such as remote control 200 shownin FIG. 2. In one embodiment, remotely controllable device 300 is aremotely controllable toy such as a remote control car or a remotecontrol plane. In other embodiments, remotely controllable device 300may be a drone, unmanned vehicle, unmanned aircraft, doll, robot, toy,and/or any other suitable device that can be controlled with a remotecontrol, such as remote control 200 shown in FIG. 2.

In various embodiments, the remotely controllable device 300 maycomprise or be in communication with fewer or additional components thanthe embodiment shown in FIG. 3. For example, in one embodiment, theremotely controllable device 300 does not comprise haptic output device340. In other embodiments, the remotely controllable device 300 may notcomprise or be in communication with a haptic output device at all.Thus, in various embodiments, the remotely controllable device 300 maycomprise or be in communication with any number of components, such asin the various embodiments disclosed herein as well as variations thatwould be apparent to one of skill in the art.

The housing 305 of the remotely controllable device 300 shown in FIG. 3provides protection for at least some of the components remotelycontrollable device 300. For example, the housing 305 may be a plasticcasing that protects the processor 310 and memory 320 from foreignarticles such as rain. In some embodiments, the housing 305 protects thecomponents in the housing 305 from damage if the remotely controllabledevice 300 is dropped by a user. The housing 305 can be made of anysuitable material including but not limited to plastics, rubbers, ormetals. Various embodiments may comprise different types of housings ora plurality of housings. For example, in some embodiments, the remotelycontrollable device 300 may be a drone, unmanned vehicle, unmannedaircraft, doll, robot, car plane, helicopter, boat, toy, and/or anyother suitable device that can be controlled with a remote control, suchas remote control 200 shown in FIG. 2.

In the embodiment shown in FIG. 3, the input and/or output devices 330provides a mechanism for the remotely controllable device 300 to receiveinformation. For example, if the remotely controllable device 300comprises a temperature sensor, then the sensor can provide informationsuch as the temperature of one or more components within the remotelycontrollable device 300 or the outside temperature or anothertemperature external to the remotely controllable device 300. As anotherexample, a remotely controllable device 300 that comprises a contactsensor 300 may be able to detect when a collision with an object such asa rock, other terrain, and/or other remotely controllable devices. Insome embodiments, the input and/or output devices 330 may comprise, beconnected with, or otherwise be in communication with one or moresensors that determine the location, pressure, a size of a contactpatch, or any of these, of one or more contacts on the input and/oroutput devices 330. For example, in one embodiment, the input and/oroutput devices 330 comprises or is in communication with a mutualcapacitance system. In another embodiment, the input and/or outputdevices 330 comprises or is in communication with an absolutecapacitance system. In some embodiments, the input and/or output devices330 may comprise or be in communication with a resistive panel, acapacitive panel, infrared LEDs, photodetectors, image sensors, opticalcameras, or a combination thereof. Thus, the input and/or output devices330 may incorporate any suitable technology to determine a contact on atouch-sensitive surface such as, for example, resistive, capacitive,infrared, optical, thermal, dispersive signal, or acoustic pulsetechnologies, or a combination thereof.

In the embodiment shown in FIG. 3, haptic output devices 340 and 360 arein communication with the processor 310 and are configured to provideone or more haptic effects. For example, in one embodiment, when anactuation signal is provided to haptic output device 340, haptic outputdevice 360, or both, by the processor 310, the respective haptic outputdevice(s) 340, 360 outputs a haptic effect based on the actuationsignal. For example, in the embodiment shown in FIG. 3, the processor310 is configured to transmit a haptic output signal to haptic outputdevice 340 comprising an analog drive signal. However, the processor 310is configured to transmit a command to haptic output device 360, whereinthe command includes parameters to be used to generate an appropriatedrive signal to cause the haptic output device 360 to output the hapticeffect. In other embodiments, different signals and different signaltypes may be sent to each of one or more haptic output devices. Forexample, in some embodiments, a processor may transmit low-level drivesignals to drive a haptic output device to output a haptic effect. Sucha drive signal may be amplified by an amplifier or may be converted froma digital to an analog signal, or from an analog to a digital signalusing suitable processors or circuitry to accommodate the particularhaptic output device being driven. A haptic output device, such ashaptic output devices 340 or 360, can be any component or collection ofcomponents that is capable of outputting one or more haptic effects.Numerous examples of haptic output devices are disclosed above andvariations are within the scope of this disclosure.

In one embodiment, haptic output device 340 and/or haptic output device360 provides status information regarding the remotely controllabledevice 300. For example, in one embodiment, the remotely controllabledevice 300 is a passenger vehicle and a haptic effect is output byhaptic output device 340 and/or haptic output device 360 when thebattery 315 of the vehicle is below a threshold level. As anotherexample, in one embodiment, the remotely controllable device 300comprises a stuffed animal, doll, or similar toy. In one suchembodiment, a user of remote control 200 could send a hug and/or a kissto a user of the stuffed animal. For example, a parent may be able touse a mobile phone to send a hug and/or a kiss to a child holding thestuffed animal. The stuffed animal can receive the hug and/or the kissand output one or more haptic effects indicating that a hug and/or kisshas been received. In another embodiment, haptic output device 340and/or haptic output device 360 provides cues from one or more remotecontrols. For example, if one remote control cedes control of theremotely controllable device to another remote control, then hapticoutput device 340 and/or haptic output device 360 can output a hapticeffect configured to indicate to a passenger of the remotelycontrollable device 300 that a new remote control is now controlling theremotely controllable device 300. As shown in FIG. 3, haptic outputdevice 360 may be coupled to or otherwise in communication with theremotely controllable device 300. In other embodiments, haptic outputdevice 360 is inside the remotely controllable device 300. In yet otherembodiments, another haptic output device, such as haptic output device260 shown in FIG. 2, is inside the remotely controllable device 300and/or coupled to or otherwise in communication with remotelycontrollable device 300. Numerous other embodiments are disclosed aboveand variations are within the scope of this disclosure.

In FIG. 3, the communication interface 350 is in communication with theprocessor 310 and provides wired or wireless communications, from theremotely controllable device 300 to other components or other devicessuch as remote control 200 shown in FIG. 2. For example, thecommunication interface 350 may provide wireless communications betweenthe remotely controllable device 300 and remote control 200. Inembodiments the communication interface 350 is capable of receivingcommands from remote control 200 shown in FIG. 2. Numerous examples ofsuitable communication interfaces are described above and variations arewithin the scope of this disclosure.

Illustrative Systems

Referring now to FIG. 4, FIG. 4 illustrates a system 400 depicting aremotely controllable device 420 and remote controls 410, 415 inaccordance with an embodiment. The system 400 shown in FIG. 4 includes aremotely controllable device 420, a first remote control 410, and asecond remote control 415. A remote control, such as remote control 410and/or remote control 415, can be any suitable device such as remotecontrol 200 discussed herein with respect to FIG. 2. A remotelycontrollable device, such as remotely controllable device 420, may beany suitable device such as remotely controllable device 300 discussedherein with respect to FIG. 3. In the embodiment shown in FIG. 4, thefirst remote control 410 is in communication with remotely controllabledevice 420 and the second remote control 415 is in communication withremotely controllable device 420. In some embodiments, the first remotecontrol 410 may be in communication with the second remote control 415.

One or more remote controls may control or otherwise operate a remotelycontrollable device. For example, in one embodiment, remote control 410may control remotely controllable device 420. In this embodiment, remotecontrol 410 may cede control of the remotely controllable device 420 toremote control 415 in response to receiving a request from remotecontrol 415 and/or remotely controllable device 420. In someembodiments, remote control 410 cedes control to remote control 415 inresponse to an event. For example, if a particular round in a game beingplayed for a user of remote control 410 ends, then remote control 410may cede control of the remotely controllable device 420 to remotecontrol 415 such that a user of remote control 415 can play the game. Asanother example, if remotely controllable device 420 receives too muchdamage while being controlled by remote control 410, then control may beswitched to remote control 415. In other embodiments, if remotelycontrollable device 420 is involved in a predetermined number of crasheswithin a predetermined period of time while being controlled by remotecontrol 410, then remote control 410 cedes control of the remotelycontrollable device 420 to remote control 415. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

In embodiments, only one remote control controls the remotelycontrollable device 420 at a time, but at various times different remotecontrols can control the remotely controllable device 420. In otherembodiments, two or more remote controls may control a remotelycontrollable device 420 at the same time. For example, remote control410 may control the direction (e.g., forward, backward, left, right,etc.) of the remotely controllable device 420 while remote control 415may control the speed of the remotely controllable device 420. Asanother example, remote control 410 may control the direction and speedof the remotely controllable device 420 while remote control 420controls the use of a real or a simulated weapon associated with theremotely controllable device 420. In yet another example, remote control410 may control the driving of a remotely controllable device 420 whileremote control 420 controls taking pictures with a camera of theremotely controllable device 420. Thus, in embodiments, operation of aremotely controllable device 420 may be shared by two, three, four, ormore remote controls.

In the embodiment shown in FIG. 4, remotely controllable device 420wirelessly communicates with remote control 410 and remote control 415through direct wireless connections. In other embodiments, one or moreremote controls and/or remotely controllable devices may be incommunication through one or more networks. A network may be anysuitable number or type of networks or links, including, but not limitedto, a dial-in network, a local area network (LAN), wide area network(WAN), public switched telephone network (PSTN), a cellular network, aWiFi network, a satellite network, the Internet, an intranet or anycombination of hard-wired and/or wireless communication links. In oneembodiment, the network is a single network. In other embodiments, thenetwork may comprise two or more networks. For example, the remotecontrol 410 may be connected to a first network and remote control 415may be connected to a second network and remotely controllable devicemay be connected to the first network and the second network. Inembodiments, remote controls, remotely controlled devices, and/or otherdevices may communicate with one or more networks over a wirelesscommunications means using Radio Frequency (RF), Bluetooth, CDMA, TDMA,FDMA, GSM, WiFi, satellite, or other cellular or wireless technology.Numerous other network configurations would be obvious to a person ofordinary skill in the art.

In one embodiment, remotely controllable device 420 is a helicopter andcomprises a camera. In this embodiment, remote control 410 controls theflying of the helicopter and remote control 415 controls the camera. Forexample, remote control 410 may be able to control the speed of apropeller of the helicopter and the direction that the helicopter istravelling. Remote control 415 can rotate the camera to view differentlocations from the helicopter and can take pictures with the camera. Asa user of remote control 410 and/or a user of remote control 415interacts with the remote control(s) to control the helicopter, remotecontrol 410 and/or remote control 415 may output haptic effects toprovide the user(s) cues regarding various events. For example, if thehelicopter crashes, then remote control 410 and/or remote control 415can output a haptic effect to indicate that a crash has occurred. Asanother example, if the speed of the propeller is below a predeterminedthreshold necessary to sustain lift, then remote control 410 and/orremote control 415 may output a haptic effect indicating that the speedof the propeller needs to be increased. In one embodiment, if thehelicopter travels above a certain altitude, then remote control 410and/or remote control 415 outputs a haptic effect indicating that thealtitude of the helicopter needs to be lowered. In another embodiment,remote control 410 and/or remote control 415 outputs a haptic effectwhen the user of remote control 415 takes a picture with the camera.

In some embodiments, the helicopter is flown outside and various hapticeffects, such as those discussed herein, are output in response toevents within and/or interactions between the helicopter and the outsideenvironment. For example, a haptic effect may be output when thehelicopter is actually flown above a certain altitude outside. Asanother example, a haptic effect may be output if the helicopterphysically crashes into the ground. In other embodiments, various hapticeffects are output in response to events with and/or interactionsbetween the helicopter and an augmented environment. For example, ahaptic effect may be output when the helicopter is flown above a certainaltitude within the augmented environment regardless of the actualphysical altitude that the helicopter is actually flying. As anotherexample, a haptic effect may be output when the helicopter crashes intoa virtual mountain shown on a display of remote control 410 and/orremote control 415, regardless of whether the helicopter physicallycrashes into an object in reality. As described herein, remote control410, remote control 415, and/or remotely controllable device 420 can beused in a real environment, in a virtual reality environment, and/or inan augmented reality. Numerous other embodiments are disclosed hereinand variations are within the scope of this disclosure.

In one embodiment, remotely controllable device 420 is a vehicle andcomprises multiple gun turrets. In this embodiment, remote control 410controls a first gun turret and remote control 420 controls a second gunturret. As a user of remote control 410 and/or a user of remote control415 interacts with the remote control(s) to control their respective gunturret, remote control 410, remote control 415, and/or remotelycontrollable device 420 may output haptic effects to provide the user(s)and/or passenger(s) cues regarding various events. For example, thevehicle may output a haptic effect indicating that a gun has been firedwhen the first gun turret and/or the second gun turret is fired. Asanother example, remote control 410 can output a haptic effect when thefirst gun turret is fired and remote control 415 can output a hapticeffect when the second gun turret is fired. In one embodiment, remotecontrol 410 outputs a haptic effect when the second gun turret is firedand remote control 415 outputs a haptic effect when the first gun turretis fired.

In some embodiments, the vehicle is driven on a road or through terrainoutside and various haptic effects, such as those discussed herein, areoutput in response to events within and/or interactions between thevehicle and the outside environment. For example, a haptic effect may beoutput when the vehicle is driven into an object, such as a rock. Asanother example, a haptic effect may be output may be output when thefirst gun turret and/or the second gun turret in the vehicle isactually, physically fired. In other embodiments, various haptic effectsare output in response to sensed events with and/or sensed interactionsbetween the vehicle and an augmented environment. For example, a hapticeffect may be output when the vehicle is driven into a virtual rockshown on a display of remote control 410 and/or remote control 415,regardless of whether the vehicle actually is driven into a physicalrock. As another example, a haptic effect may be output when the firstgun turret and/or the second gun turret is virtually fired in anaugmented reality, regardless of whether the first gun turret and/or thesecond gun turret is actually fired in reality. As described herein,remote control 410, remote control 415, and/or remotely controllabledevice 420 can be used in a real environment, in a virtual realityenvironment, and/or in an augmented reality. Numerous other embodimentsare disclosed herein and variations are within the scope of thisdisclosure.

In another embodiment, remotely controllable device 420 comprises astuffed animal, doll, or similar toy. In one such embodiment, a usersuch as a parent can control a stuffed animal using remote control 415.For example, a user may be able to use a smartphone to control thestuffed animal. In this embodiment, the user of the smartphone can sendmessages to a user of the stuffed animal which are output as hapticeffects. For example, a parent using a smartphone or other suitableremote control 415 can send a hug or a kiss to a user (such as a child)holding or otherwise contacting the remotely controllable device 420.Thus, if the remotely controllable device 420 is a stuffed animal, thenthe stuffed animal can receive the message from the smartphone andoutput a haptic effect configured to indicate that a message (such as akiss or a hug) has been received. If the child is holding the stuffedanimal, then the child may feel the haptic effects output by the stuffedanimal and know that he or she has been sent a hug and/or a kiss. Insome embodiments, the stuffed animal or other toy may also transmithaptic messages back to the remote control device, such as to indicatethat the hug or kiss is being received. In other embodiments, theremotely controllable device may be a robotic toy or any other suitableremotely controllable device. Numerous other embodiments are disclosedherein and variations are within the scope of this disclosure.

As another example, remotely controllable device 420 may be a stuffedanimal, such as a cat. In one embodiment, remote control 410 comprisesan application having a graphical user interface including an image ofan animal corresponding to the stuffed animal. For example, if thestuffed animal is a cat, then the graphical user interface on the remotecontrol 410 may display an image of a cat. In one embodiment, as a userinteracts with the image of the cat displayed on the remote control 410,one or more haptic effects are output by the remotely controllabledevice 420. For example, if the remote control 410 comprises atouchscreen display, and a user of the remote control 410 pets the imageof the cat displayed on the touchscreen display, then the remotelycontrollable device 420 (e.g., cat) may output a haptic effect thatfeels like a purr. In some embodiments, the remote control 410 mayoutput a haptic effect that feels like a purr when a user of the remotecontrol 410 pets the image of the cat displayed on the touchscreendisplay. In another embodiment, one or more haptic effects may be outputby remote control 410 when a user interacts with the remotelycontrollable device 420 (e.g., cat). For example, as a user pets thecat, remote control 410 may output a haptic effect that feels like apurr. In some embodiments, the remotely controllable device 420 mayoutput a haptic effect that feels like a purr when a user of theremotely controllable device 420 pets the cat. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

Referring now to FIG. 5, FIG. 5 illustrates a system 500 depictingremotely controllable devices 520, 525 and remote controls 510, 515 inaccordance with an embodiment. The system 500 shown in FIG. 5 includes afirst remotely controllable device 520, a second remotely controllabledevice 525, a first remote control 510, a second remote control 515, anda computer 540, though embodiments are not limited to only two remotecontrols and two remotely controllable devices.

A remote control, such as remote control 510 and/or remote control 515,can be any suitable device such as remote control 200 discussed hereinwith respect to FIG. 2. A remotely controllable device, such as remotelycontrollable device 520 and/or remotely controllable device 525, may beany suitable device such as remotely controllable device 300 discussedherein with respect to FIG. 3. In the embodiment shown in FIG. 5, thefirst remote control 510 is in communication with remotely controllabledevice 520 and the network 530, and the second remote control 515 is incommunication with remotely controllable device 525 and the network 530.In some embodiments, the first remote control 510 is in communicationwith the second remote control 515, the first remotely controllabledevice 520, the second remotely controllable device 525, and/or computer540 directly, such as by direct wired or wireless communication, and/orthrough the network 530. In some embodiments, the second remote control515 is in communication with the first remote control 510, the firstremotely controllable device 520 and/or the second remotely controllabledevice 525 directly and/or through the network 530. In otherembodiments, remotely controllable device 520 is in communication withremotely controllable device 525 directly and/or through the network530.

In the system 500 shown in FIG. 5, the computer 540 is connected to thenetwork 530. In this embodiment, the computer 540 can performcoordination functions between remote control 510, remote control 515,remotely controllable device 520, and/or remotely controllable device525. For example, the computer 540 may track the amount of damage thatremotely controllable device 520 and/or remotely controllable device 525has incurred. As another example, the computer 540 keep score of a gamebeing played between a user of remote control 510 and a user of remotecontrol 515. In embodiments, the computer 540 can transmit the amount ofdamage, scores, and/or other information regarding gameplay events, userinformation, status information, historical information from previousgameplay, etc. to remote control 510, remote control 515, remotelycontrollable device 520, and/or remotely controllable device 525.Computer 540 may be a general purpose computer or a specificallydesigned electronic device, such as a web server. In the embodimentshown in FIG. 5, computer 540 is a single computer. In otherembodiments, computer 540 may be in communication with a plurality ofother devices, such as a plurality of servers. In some embodiments,computer 540 is part of or in communication with a content distributionnetwork (CDN). In other embodiments, system 500 may not include acomputer 540. Numerous other embodiments are disclosed herein andvariations are within the scope of this disclosure.

In embodiments, computer 540 maintains a virtual and/or augmentedreality for one or more of the remote controls and/or remotelycontrollable devices. For example, the computer 540 may maintain anaugmented reality by stitching together or otherwise combininginformation from various remote controls and/or remotely controllabledevices into a single environment. In this embodiment, absolutemovements of each device at their own location are translated intorelative movements in the augmented reality and the remote controls areprovided with information about its own remotely controllable device aswell as other devices within the augmented related. For example, atleast a portion of the augmented reality can be displayed on a displayof a remote control based at least in part on the information receivedfrom the computer 540. In embodiments, users of the remote controls caninteract with each other through am augmented environment and/or avirtual environment. For example, users may be able to have car races ormock helicopter battles in various augmented environments. As anotherexample, in one embodiment, projectile movement of a weapon is simulatedby computer 540 when a user on a remote control presses a trigger buttonindicating that a weapon should be fired. In this embodiment, theprojectile movement simulated by computer 540 may be displayed on adisplay of one or more of the remote controls. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

One or more remote controls may control or otherwise operate at least aportion of one or more remotely controllable devices. For example, inone embodiment, remote control 510 may control at least a portion ofremotely controllable device 520 and/or remotely controllable device525. In this embodiment, remote control 510 may cede control of at leasta portion of one or more of the remotely controllable device 520, 525 toremote control 515. Thus, in embodiments, one remote control controls atleast a portion of remotely controllable device 520 and another remotecontrol controls another portion of remotely controllable device 525. Inother embodiments, each remote control may control at least a portion ofseparate remotely controllable devices. For example, remote control 510may control at least a portion of remotely controllable device 520 andremote control 515 may control at least a portion of remotelycontrollable device 525.

One or more remote controls may control or otherwise operate at least aportion of one or more remotely controllable devices. For example, inone embodiment, remote control 510 may control at least a portion ofremotely controllable device 520 and/or remotely controllable device525. In this embodiment, remote control 510 may cede control of at leasta portion of one or more of the remotely controllable devices 520, 525to remote control 515 in response to receiving a request from remotecontrol 515, remotely controllable device 520, and/or remotelycontrollable device 525. In some embodiments, remote control 510 cedescontrol to remote control 515 in response to an event. For example, if aparticular round in a game being played for a user of remote control 510ends, then remote control 510 may cede control of remotely controllabledevice 520 and/or remotely controllable device 525 to remote control 515such that a user of remote control 515 can play the game. As anotherexample, if remotely controllable device 520 receives too much damagewhile being controlled by remote control 510, then control may beswitched to remote control 515. In other embodiments, if remotelycontrollable device 520 is involved in a predetermined number of crasheswithin a predetermined period of time while being controlled by remotecontrol 510, then remote control 510 cedes control of the remotelycontrollable device 520 to remote control 515. In another embodimentremote control 510 initially controls remotely controllable device 520and remotely controllable device 525 and control is ceded to remotecontrol 515 if communication between remote control 510 and remotelycontrollable device 525 is lost. Numerous other embodiments aredisclosed herein and variations are within the scope of this disclosure.

In embodiments, two or more remote controls may control one or moreremotely controllable devices at the same time. For example, remotecontrol 510 may control the direction (e.g., forward, backward, left,right, etc.) of the remotely controllable device 520 while remotecontrol 515 may control the speed of the remotely controllable device520. As another example, remote control 510 may control the direction ofthe remotely controllable device 520 while remote control 515 controlsthe use of a real or a simulated weapon associated with the remotelycontrollable device 520. In some embodiments, additional remote controlsmay be used to control portions of a remotely controllable device. Forexample, a third remote control (not shown) may be used to control acamera associated with remotely controllable device 520. Thus, inembodiments, operation of one or more remotely controllable devices,such as remotely controllable device 520 and/or remotely controllabledevice 525, may be shared by two, three, four, or more remote controls.

Various types of information can be sent and/or received between one ormore remote controls and one or more remotely controllable devices. Forexample, sensor information from remotely controllable device 520 and/orremotely controllable device 525 can be sent to remote control 510and/or remote control 515. As another example, information such as avideo and/or audio can be sent to remote control 510 and/or remotecontrol 515. In some embodiments, information can be sent to and/from aremote control to one or more other remote controls and/or one or moreremotely controllable devices. For example, information regarding thelocation of one remotely controllable device may be sent from theremotely controllable device and/or a remote control in communicationwith the remotely controllable device to other remote controls and/orother remotely controllable devices. In some embodiments, where one ormore remote controls comprise or are in communication with a display,information received from one or more remote controls and/or one or moreremotely controllable devices may be displayed on the display. Forexample, a location, speed, direction, temperature, video, other sensorinformation, etc. may be displayed on the display of a remote control.

In some embodiments, one or more graphical images are overlaid withinformation received from another device. For example, referring to FIG.5, remote control 510 may display a location of remotely controllabledevice 520 and a location of remotely controllable device 525 based oninformation received from remotely controllable device 520 and frominformation received from remote control 515 that is in communicationwith remotely controllable device 525. In addition, video feeds fromremotely controllable device 520 and/or remotely controllable device 525may be sent to remote control 510 and/or remote control 515. In thisembodiment, one or more graphics may be overlaid on one or more of thevideo feeds. For example, if the remotely controllable devices 520, 525are RC cars, then remote control 510 and/or remote control 515 mayoverlay graphics on one or more video feeds from the RC car(s) tosimulate racing at a race track. Numerous other examples of informationthat can be sent to and/or from remote controls and/or remotelycontrollable devices are disclosed herein and variations are within thescope of this disclosure.

In the embodiment shown in FIG. 5, remotely controllable device 520wirelessly communicates with remote control 510 through a directwireless connection. In other embodiments, one or more remote controlsand/or remotely controllable devices may be in communication through oneor more networks, such as network 530. Network 530 may be any suitablenumber or type of networks or links, including, but not limited to, adial-in network, a local area network (LAN), wide area network (WAN),public switched telephone network (PSTN), a cellular network, a WiFinetwork, a satellite network, the Internet, an intranet or anycombination of hard-wired and/or wireless communication links. In oneembodiment, the network is a single network. In other embodiments, thenetwork 530 may comprise two or more networks. For example, the remotecontrol 510 may be connected to a first network and remote control 515may be connected to a second network and remotely controllable device520 may be connected to the first network and the second network 525. Inembodiments, remote controls, remotely controlled devices, and/or otherdevices may communicate with one or more networks over a wirelesscommunications means using Radio Frequency (RF), Bluetooth, CDMA, TDMA,FDMA, GSM, WiFi, satellite, or other cellular or wireless technology.Numerous other network configurations would be obvious to a person ofordinary skill in the art.

In one embodiment, remotely controllable device 520 and remotelycontrollable device 525 are each helicopters. In this embodiment, remotecontrol 510 controls remotely controllable device 520 and remote control515 controls remotely controllable device 525. For example, remotecontrol 510 may be able to control the driving of remotely controllabledevice 520 and remote control 515 can control the driving of remotelycontrollable device 525. As a user of remote control 510 and/or a userof remote control 515 interacts their respective remote control tocontrol the helicopters, remote control 510 and/or remote control 515may output haptic effects to provide the user(s) with cues regardingvarious events. For example, if remotely controllable device 520collides with remotely controllable device 525, then remote control 510and/or remote control 515 may output a haptic effect to indicate that acrash has occurred. As another example, if remotely controllable device520 is approaching remotely controllable device 525, then remote control510 and/or remote control 515 may output a haptic effect to indicatethat remotely controllable device 520 is approaching remotelycontrollable device 525.

In some embodiments, remotely controllable device is physically in thesame location as remotely controllable device 525. For example, remotelycontrollable device 520 and remotely controllable device 525 may both beflown in the same park or in the same yard. In this embodiment, varioushaptic effects, such as those discussed herein, are output in responseto events within and/or interactions between remotely controllabledevice 520, remotely controllable device 525, and/or the outsideenvironment. For example, a haptic effect may be output when remotelycontrollable device 520 physically crashes into remotely controllabledevice 525. As another example, a haptic effect may be output if bothremotely controllable device 520 and remotely controllable device 525physically are flown within a predetermined distance of each other.

In other embodiments, various haptic effects are output in response toevents with and/or interactions between remotely controllable device520, remotely controllable device 525, and/or an augmented environment.For example, a haptic effect may be output when remotely controllabledevice 520 and/or remotely controllable device 525 is flown above acertain altitude within the augmented environment regardless of theactual physical altitude that the remotely controllable device 520and/or remotely controllable device 525 is actually flying. As anotherexample, a haptic effect may be output when remotely controllable device520 and remotely controllable device 525 crash into each other in theaugmented environment regardless of whether an actual, physical crashbetween the devices 520, 525 occurs. In one embodiment, a haptic effectis output when remotely controllable device 520 crashes into a virtualobject, such as a virtual rock, shown on a display of remote control 510and/or remote control 515, regardless of whether remotely controllabledevice 520 physically crashes into an object in reality. As describedherein, remote control 510, remote control 515, remotely controllabledevice 520, and/or remotely controllable device 525 can be used in areal environment, in a virtual reality environment, and/or in anaugmented reality environment. Numerous other embodiments are disclosedherein and variations are within the scope of this disclosure.

In one embodiment, remotely controllable device 520 is a helicopter andcomprises a gun and remotely controllable device 525 is a vehicle andcomprises a gun. In this embodiment, remote control 510 controls theflying of the helicopter and another remote control (not shown) controlsthe gun of the helicopter. In addition, in this embodiment, remotecontrol 515 controls the driving of the vehicle and another remotecontrol (not shown) controls the gun of the vehicle. As users of thevarious remote controls interact with their respective remote control tocontrol the helicopter, vehicle, and/or guns, haptic effects can beoutput by one or more of the remote controls to provide the user(s) cuesregarding various events. For example, if a user of the remote controlcontrolling the gun of the helicopter interacts with the remote controlto fire the gun at the car, then a haptic effect may be output by remotecontrol 510, remote control 515, and/or other remote controls toindicate that a shot has been fired. As another example, if the vehicleis hit by a bullet fired by the gun of the helicopter, then remotecontrol 510, remote control 515, and/or other remote controls may outputone or more haptic effects to indicate that the vehicle has beendamaged. In one embodiment, if the helicopter is approaching the vehiclewithin a predetermined distance, then remote control 510, remote control515, and/or other remote controls may output one or more haptic effectsto indicate that the helicopter is approaching.

In some embodiments, the helicopter and the vehicle are both physicallyin the same location. For example, the helicopter and the vehicle mayboth physically be in the same park or in the same yard. In thisembodiment, various haptic effects, such as those discussed herein, areoutput in response to events within and/or interactions between thehelicopter, vehicle, and/or physical environment. For example, a hapticeffect may be output if the helicopter crashes into the ground or if thevehicle overturns. As another example, a haptic effect can be output ifthe helicopter crashes into the vehicle. In one embodiment, a hapticeffect is output if the helicopter and the vehicle come within apredetermined distance of each other. As another example, a hapticeffect may be output if a bullet fired by a gun of the helicopterphysically hits the vehicle.

In other embodiments, various haptic effects are output in response toevents with and/or interactions between the helicopter, the vehicle,and/or an augmented environment. For example, the helicopter and thevehicle may be physically operated in the same or in differentlocations. Information from these devices can be received by remotecontrol 510, remote control 515 and/or other remote controls andcombined or otherwise incorporated into a virtual environment to createan augmented environment. For example, movements of the helicopterand/or the vehicle can be received by the remote controls andincorporated into an augmented environment. Thus, if a remote control isoperated to fly the helicopter to the right in a physical environment,then the helicopter can flown to the right in an augmented environmentby updating a display of a remote control to reflect that the helicopteris moving to the right in the augmented environment.

Various haptic effects, such as those discussed herein, may be output inresponse to events within and/or interactions between the helicopter,vehicle, physical environment, and/or augmented environment. Forexample, a haptic effect may be output if the helicopter crashes intothe ground in the augmented environment regardless of whether thehelicopter physically crashes into the ground in the physicalenvironment. In one embodiment, a haptic effect is output if thehelicopter physically crashes into an object in the physical environmentregardless of whether the helicopter crashes into an object in theaugmented environment. As another example, a haptic effect can be outputif the helicopter and the vehicle come within a predetermined distanceof each other in the augmented environment regardless of the actualphysical distance between the helicopter and the vehicle in reality. Inone embodiment, a haptic effect is output if a virtual or real bulletfired by the helicopter hits the vehicle in an augmented realityenvironment. As described herein, remote control 510, remote control515, remotely controllable device 520, remotely controllable device 525,other remote control(s), other remotely controllable device(s), otherdevices, or a combination thereof, can be used in a real environment, ina virtual environment, and/or in an augmented reality environment.Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

In embodiments, remotely controllable device 520 and remotelycontrollable device 525 are in a same location, such as in a samebuilding, in a same park, on the same road, etc. In other embodiments,remotely controllable device 520 and remotely controllable device 525are in different physical locations. For example, remotely controllabledevice 520 may be located indoors and remotely controllable device 525may be located outdoors. As another example, remotely controllabledevice 520 may be located in one city and remotely controllable device520 may be located in another city. In some embodiments, remote control510 and remote control 515 are in a same location, such as in a samebuilding, in a same park, etc. In other embodiments, remote control 510and remote control 515 are in different physical locations. For example,remote control 510 may be located indoors and remotely control 515 maybe located outdoors. As another example, remote control 510 may belocated in one city and remote control 515 may be located in anothercity. Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

Illustrative Method of Generating Haptic Effects

Referring now to FIG. 6, this figure illustrates a flow chart directedto a method 600 of generating haptic effects in accordance with anembodiment. The method 600 shown in FIG. 6 will be described withrespect to the remote control 200 shown in FIG. 2 and the remotelycontrollable device 300 shown in FIG. 3. In embodiments, the method 600can be performed by one or more of the devices shown in system 100 inFIG. 1, system 400 in FIG. 4, and/or system 500 in FIG. 5. However, themethod 600 shown in FIG. 6 will be described with respect to system 500shown in FIG. 5. In addition, aspects of method 600 will be describedwith respect to remote control 200 shown in FIG. 2; however, otherdevices such as remotely controllable device 300 shown in FIG. 3 mayperform these functions in various embodiments.

The method 600 begins in block 610 when a remote control and/or aremotely controllable device receives information from another remotecontrol and/or another remotely controllable device. In one embodiment,referring to FIG. 5, remote control 510 receives sensor informationindicating a speed and direction of remotely controllable device 520from remotely controllable device 520. Information may be sent to remotecontrol 510 in response to a request for the information sent by remotecontrol 510 to remotely controllable device 520. In another embodiment,the information may be pushed from remotely controllable device 520 toremote control 510. For example, remotely controllable device 520 maysend information to remote control 510 without remote control 510requesting the information. In other embodiments, remote control 510 mayreceive information directly or through one or more networks, such asnetwork 530, from remote control 515, remotely controllable device 525,other remote controls, other remotely controllable devices, otherelectronic devices, or a combination thereof.

The information received by a remote control 200 and/or a remotelycontrollable device 300 can include information associated with one ormore accelerometers, gyroscopes, digital compasses, sensors usable todetermine location, pressure, speed, wind speed, temperature, forceand/or size, resistive panels, capacitive panels, infrared LEDs,photodetectors, image sensors, optical cameras, other cameras,microphones, speakers. The information can be received by anapplication, an applet, a plug-in, or a script being executed by aprocessor on remote control 200 and/or remotely controllable device 300.Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

Referring back to method 600, once the remote control and/or remotelycontrollable device has received the information 610, the method 600proceeds to block 620. In block 620, the remote control and/or theremotely controllable device determines one or more haptic effects 620.For example, in an embodiment, remote control 510 receives sensorinformation indicating a speed and direction of remotely controllabledevice 520 from remotely controllable device 520. In this embodiment,remote control 510 determines a haptic effect based at least in part onthe received speed and/or the direction. For example, remote control 510may determine based at least in part on the speed and direction that theremotely controllable device 520 is approaching an object. In thisembodiment, the remote control 510 determines a haptic effect configuredto warn a user of the remote control that the remotely controllabledevice 520 is approaching an object.

One or more haptic effects may be determined by a remote control 200and/or a remotely controllable device 300 in any number of ways. In oneembodiment, one or more haptic effects are determined by a remotecontrol 200 based at least in part on information received from aremotely controllable device 300. For example, remote control 200 maydetermine a haptic effect when sensor information received from aremotely controllable device 300 indicates that the remotelycontrollable device 300 is approaching an object. As another example,remote control 200 may determine a haptic effect when sensor informationreceived from a remotely controllable device 300 indicates that theremotely controllable device 300 has collided with another object. Insome embodiments, remote control 200 determines a haptic effect based atleast in part on state information associated with a remotelycontrollable device 300. For example, if the remote control 200 receivesinformation from the remotely controllable device 300 that indicatesthat an engine associated with the remotely controllable device 300 hasstalled, then the remote control 200 may determine a haptic effectconfigured to indicate to a user of the remote control 200 that theengine of the remotely controllable device 300 has stalled. Inembodiments, a haptic effect may be determined by remote control 200based at least in part on augmented reality. For example, if theremotely controllable device 300 is a remote control car, then theremote control 200 may display a virtual race track and the remotecontrol car on a display associated with the remote control 200. In thisembodiment, if the remote control car collides with a virtual car in theaugmented reality environment, then the remote control 200 determines ahaptic effect configured to indicate that a collision has occurred. Asanother example, the remote control 200 may determine a haptic effect asthe remote control car approaches a virtual car on the race track.Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

In some embodiments, one or more haptic effects are determined based atleast in part on information from another remote control. For example, afirst remote control 200 may control the driving a remotely controllabledevice 300 and a second remote control 200 may control a cameraassociated with the remotely controllable device 300. Thus, inembodiments, operational control of a remotely controllable device 300may be shared between two, three, four, or more remote controls. In thisembodiment, one or more haptic effects may be determined by the firstremote control based at least in part on the second remote control. Forexample, the first remote control may receive a request from the secondremote control and/or the remotely controllable device 300 to cedecontrol of at least a portion of the operation to the second remotecontrol. In this embodiment, the first remote control may determine ahaptic effect indicating that at least partial control of the remotelycontrollable device 300 is being ceded to the second remote control. Forexample, in one embodiment, a first controller cedes control to a secondcontroller based on the remotely controllable device being out of rangeof the first controller but still in range of the second controller. Asanother example, a first controller may cede control to a secondcontroller in response to a user selecting a button to switchcontrollers. In one embodiment, the second remote control may determinea haptic effect configured to indicate a collision when a user of thefirst remote control drives the remotely controllable device 300 into areal or virtual object.

In some embodiments, one or more haptic effects are determined based atleast in part on interactions between two or more remote controls and/ortwo or more remotely controllable devices. For example, referring toFIG. 5, a first remote control 510 may operate remotely controllabledevice 520 in a first location and a second remote control 515 mayoperate remotely controllable device 525 in a second location. Thesecond location may be remote from the first location. In thisembodiment, communication between remote control 510 and remotelycontrollable device 520 may occur directly while communication betweenremote control 510 and remote control 515 occurs through network 530. Inthis embodiment, remote control 510 may determine one or more hapticeffects based at least in part on information from remotely controllabledevice 520 and/or remotely controllable device 525. For example, remotecontrol 510 may determine a haptic effect indicating a collision whenremotely controllable device 520 collides with remotely controllabledevice 525 in an augmented reality view displayed on a display of theremote control 510. As another example, remote control 510 may determinea haptic effect configured to indicate that a weapon has been fired whena user of remote control 520 provides input to remote control 520indicating that a weapon of remotely controllable device 525 should befired. Numerous other embodiments of when and how haptic effects may bedetermined are disclosed herein, such as with respect to system 100,system 400, and/or system 500, and variations are within the scope ofthis disclosure.

In embodiments, one or more haptic effects may be based at least in parton information received from a remote control 200, such as the variousinformation described above with respect to FIG. 2, and/or informationreceived from a remotely controllable device 300, such as the variousinformation described herein with respect to FIG. 3. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

Referring back to method 600, once the remote control and/or theremotely controllable device determines one or more haptic effects 620,the method 600 proceeds to block 630. In block 630, a signal isgenerated. For example, in an embodiment where an event involves acollision between a remotely controllable device and another object—suchas the ground, a rock, a building, another remotely controllable device,etc.—then a signal may be generated when the collision between theremotely controllable device and the object occurs. In one embodiment, acollision physically occurs. For example, a remotely controllable devicemay physically contact a rock on the ground. As another example, aremotely controllable device may physically contact another remotelycontrollable device. In some embodiments, a collision occurs inaugmented reality. For example, a collision may occur when a remotelycontrollable device contacts an augmented rock being displayed on adisplay of the remote control. As another example, a collision may occurwhen a remotely controllable device virtually contacts another remotelycontrollable device. In one embodiment, a remotely controllable deviceis struck by a gunshot from another remotely controlled device. Thegunshot may be from a real, physical bullet or from a virtual bullet. Inthis embodiment, haptic effect may be determined and a signal generatedto indicate that a remotely controllable device has been hit. Thus, invarious embodiments, interactions between multiple remotely controllabledevices can be detected by a remotely controllable device. Theinteractions can be reported to one or more remote controls by aremotely controllable device.

A remotely controllable device may be in communication with one, two,three, or more remote controls. In such embodiments, a remotelycontrollable device may be controlled by one, two, three, or more remotecontrols. In some embodiments, one or more haptic effects are determinedbased at least in part on environmental feedback from one or more of theremotely controllable toys, gameplay events from one or more remotecontrols, gameplay events from one or more interactions between one ormore of the remotely controllable devices, or a combination thereof. Anydetermined haptic effect(s) may be output to any number of remotecontrols, remotely controllable devices, other devices, or a combinationthereof.

Multiple remotely controllable devices and/or remote controls may be incommunication with each other over one or more networks. In suchembodiments, one or more haptic effects may be determined based at leastin part on multiplayer interactions between the remotely controllabledevices and/or the remote controls. For example, if sensor informationreceived from one remotely controllable device indicates that acollision has occurred, then one or more of remote controls may output ahaptic effect configured to indicate that a collision has occurred.

In one embodiment, a remote control comprises a display and a remotelycontrollable device comprises a camera. In such an embodiment, theremote control can display information from one or more of the remotelycontrollable devices. For example, at least a portion of one or morevideo feeds from one or more remotely controllable devices may bedisplayed on a display of a remote control. A video feed from one ormore remotely controllable devices can be overlaid with other graphicalimages and displayed on the display of the remote control. Thus, inembodiments, an augmented reality using information received from one ormore remotely controllable devices and/or one or more remote controls isdisplayed on a display of a remote control. In addition, haptic effectscan be determined based at least in part on the augmented reality,environmental feedback, gameplay events, state information of one ormore devices, other information, or a combination thereof. Anydetermined haptic effect(s) may be output to any number of remotecontrols, remotely controllable devices, other devices, or a combinationthereof. Numerous other embodiments are disclosed herein and variationsare within the scope of this disclosure.

In an embodiment, a signal is generated the first time an event occurs.For example, if the event comprises a collision between a remotelycontrollable device and an object, then the first time that a collisionbetween the remotely controllable device and the object occurs, theprocessor 210 generates a signal. In one embodiment, if a subsequentcollision occurs and/or if a subsequent collision occurs within apredetermined period of time between the remotely controllable deviceand the object, then another signal is not generated. In otherembodiments, if a subsequent collision occurs between the remotelycontrollable device and the object, then the processor 210 generates asignal based on the subsequent collision.

In one embodiment, a signal is generated each time an event occurs.Thus, referring to the example above, each time a collision occursbetween the remotely controllable device and the object, the processor210 generates a signal. Therefore, if the remotely controllable devicecollides with a rock and then collides again with the rock for a secondtime, then the processor 210 would generate a signal twice. In anotherembodiment, a signal is generated only the first time the event occurs.

In embodiments, one or more signals are generated at any number of timesbased at least in part on information received from a remotelycontrollable device, a remote control, user input, other devices, or acombination thereof. The information can include environmentalinformation such as a temperature, wind speed, terrain conditions (e.g.,water, mud, dry land, mountains, hills, etc.), information regarding thestate of a device, information regarding the proximity of a device toanother device, information regarding events of one or more users of oneor more remote controls, information regarding events initiated by auser's own interaction with a remote control, information regardingevents received from another remote control being operated by a seconduser, other suitable information, or a combination thereof. Informationcan be information corresponding to real life conditions such as anactual temperature and/or information corresponding to virtualconditions such as a remotely controllable device colliding with avirtual rock. In one embodiment, one or more signals are generated whenan event occurs. In some embodiments, one or more signals are generatedprior to an event occurring. In other embodiments, one or more signalsare generated after an event occurs. Numerous other embodiments aredisclosed herein and variations are within the scope of this disclosure.

In some embodiments, the processor 210 generates a single signal whenthe event occurs. For example, in one embodiment, the processor 210generates a signal configured to cause a haptic output device, such ashaptic output device 240 or haptic output device 260, to output a hapticeffect. The haptic effect may indicate that an object is currentlydisplayed on the display 230, that an object is about to be displayed onthe display 230, that an object is approaching, that an event hasoccurred, that an event is about to occur, or a combination thereof.

In other embodiments, the processor 210 generates two, three, or moresignals. For example, in one embodiment, the processor 210 generates afirst signal configured to cause a first haptic effect and a secondsignal configured to cause a second haptic effect. In some embodiments,the processor 210 generates a different signal for each event thatoccurs. In various embodiments, the processor 210 generates one or moresignals configured to cause the touch-sensitive display 230, thecommunication interface 250, the haptic output device 240, the hapticoutput device 260, the speaker 270, other components of the device 200,other components of devices in communication with the device 200, or acombination thereof to output one or more of the generated signals, suchas a video signal, audio signal, haptic output signal, and/or acommunication signal. For example, in one embodiment, the processor 210generates a signal when the event occurs where the signal is configuredto cause a haptic output device in another device to cause a hapticeffect. In one embodiment, the processor 210 sends the signal to theother device through the communication interface 250.

In one embodiment, a generated signal includes a command for a device orcomponent to perform a specified function, such as to output a hapticeffect or transmit a message to a remote device. In another embodiment,a generated signal includes parameters which are used by a device orcomponent receiving the command to determine a response or some aspectof a response. Parameters may include various data related to, forexample, magnitudes, frequencies, durations, or other parameters that ahaptic output device can use to determine a haptic effect, output ahaptic effect, or both. For example, in one embodiment, the processor210 generates a signal configured to cause haptic output device 240 tooutput a haptic effect. In such an embodiment, the signal may include apressure parameter that the haptic output device 240 uses to determinethe intensity of the haptic effect to output. For example, according toone embodiment, the larger the pressure parameter the haptic outputdevice 240 receives, the more intense the haptic effect that is output.

In one embodiment, an intensity parameter is used by a haptic outputdevice to determine the intensity of a haptic effect. In thisembodiment, the greater the intensity parameter, the more intense thehaptic effect that is output. In one embodiment, the intensity parameteris based at least in part on sensor information, such as speed,direction, etc., of a remotely controllable device when an event occurs.Thus, according to one embodiment, a larger intensity parameter is sentto a haptic output device when an event occurs while the remotelycontrollable device is travelling at a faster speed than when an eventoccurs while the remotely controllable device is travelling at a slowerspeed. A signal may include data that is configured to be processed by ahaptic output device, display, communication interface, speaker, orother components of a device or in communication with a device in orderto determine an aspect of a particular response.

It will be recognized that any type of input synthesis method may beused to generate the interaction parameter for one or more haptic effectsignals including, but not limited to, the method of synthesis exampleslisted in TABLE 1 below. A drive signal may be applied to a hapticactuator according to the interaction parameter. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

Referring again to FIG. 6, once a signal has been generated as specifiedin block 630, the next step of method 600 is to output the signal asshown in block 640. For example, in one embodiment, the processor 210generated a first signal configured to cause haptic output device 240 tooutput a haptic effect. In such an embodiment, the processor 210 outputsthe signal to haptic output device 240. As another example, in anembodiment, the processor 210 generated a first haptic output signalconfigured to cause haptic output device 240 to output a first hapticeffect and generated a second haptic output signal configured to causehaptic output device 260 to output a second haptic effect. In thisembodiment, the processor 210 outputs the first haptic output signal tohaptic output device 240 and the second haptic output signal to hapticoutput device 260.

In various embodiments, the processor 210 may output one or moregenerated signals to any number of devices. For example, the processor210 may output one signal to the communication interface 250. In oneembodiment, the processor 210 may output one generated signal to thetouch-sensitive display 230, another generated signal to thecommunication interface 250, and another generated signal to the hapticoutput device 260. In other embodiments, the processor 210 may output asingle generated signal to multiple components or devices. For example,in one embodiment, the processor 210 outputs one generated signal toboth haptic output device 240 and haptic output device 260. In anotherembodiment, the processor 210 outputs one generated signal to hapticoutput device 240, haptic output device 260, and communication interface250. In still another embodiment, the processor 210 outputs onegenerated signal to both haptic output device 240 and haptic outputdevice 260 and outputs a second generated signal to the touch-sensitivedisplay 230.

As discussed above, the processor 210 may output one or more signals tothe communication interface 250. For example, the processor 210 mayoutput a signal to the communication interface 250 instructing thecommunication interface 250 to send data to another component or devicein communication with the device 200. In such an embodiment, thecommunication interface 250 may send data to the other device and theother device may perform a function such as updating a displayassociated with the other device or the other device may output a hapticeffect. Thus, in embodiments, a second device may output a haptic effectbased at least in part upon an interaction with a first device incommunication with the second device. In other embodiments, a seconddevice may perform any number of functions such as, for example,updating a display associated with the second device or outputting asound to a speaker associated with the second device based at least inpart on an interaction with a first remote control 200.

In various embodiments, after the processor 210 outputs a signal to acomponent, the component may send the processor 210 a confirmationindicating that the component received the signal. For example, in oneembodiment, haptic output device 260 may receive a command from theprocessor 210 to output a haptic effect. Once haptic output device 260receives the command, the haptic output device 260 may send aconfirmation response to the processor 210 that the command was receivedby the haptic output device 260. In another embodiment, the processor210 may receive completion data indicating that a component not onlyreceived an instruction but that the component has performed a response.For example, in one embodiment, haptic output device 240 may receivevarious parameters from the processor 210. Based on these parametershaptic output device 240 may output a haptic effect and send theprocessor 210 completion data indicating that haptic output device 240received the parameters and outputted a haptic effect.

It will be recognized that any type of input synthesis method may beused to generate the interaction parameter for one or more haptic effectsignals including, but not limited to, the method of synthesis exampleslisted in TABLE 1 below. A drive signal may be applied to a hapticactuator according to the interaction parameter. Numerous otherembodiments are disclosed herein and variations are within the scope ofthis disclosure.

Table 1—Methods of Synthesis

Additive synthesis—combining inputs, typically of varying amplitudes

Subtractive synthesis—filtering of complex signals or multiple signalinputs

Frequency modulation synthesis—modulating a carrier wave signal with oneor more operators

Sampling—using recorded inputs as input sources subject to modification

Composite synthesis—using artificial and sampled inputs to establish aresultant “new” input

Phase distortion—altering the speed of waveforms stored in wavetablesduring playback

Waveshaping—intentional distortion of a signal to produce a modifiedresult

Resynthesis—modification of digitally sampled inputs before playback

Granular synthesis—combining of several small input segments into a newinput

Linear predictive coding similar technique as used for speech synthesis

Direct digital synthesis—computer modification of generated waveforms

Wave sequencing—linear combinations of several small segments to createa new input

Vector synthesis—technique for fading between any number of differentinput sources

Physical modeling—mathematical equations of the physical characteristicsof virtual motion

Illustrative Method of Generating Haptic Effects

Referring now to FIG. 7, this figure illustrates a flow chart directedto a method 700 of generating haptic effects in accordance with anembodiment. The method 700 shown in FIG. 6 will be described withrespect to the remote control 200 shown in FIG. 2 and the remotelycontrollable device 300 shown in FIG. 3. In embodiments, the method 700can be performed by one or more of the devices shown in system 100 inFIG. 1, system 400 in FIG. 4, and/or system 500 in FIG. 5. However, themethod 700 shown in FIG. 7 will be described with respect to system 500shown in FIG. 5. In addition, aspects of method 700 will be describedwith respect to remote control 200 shown in FIG. 2; however, otherdevices such as remotely controllable device 300 shown in FIG. 3 mayperform these functions in various embodiments.

The method 700 begins in block 710 when information is received from afirst remote control. For example, referring to FIG. 5, computer 540 mayreceive information from remote control 510 through network 530. In oneembodiment, a user operates remote control 510 in a manner that causesan action to occur in remotely controllable device 520. For example, theuser may press a button on the remote control 510 to fire a weapon onremotely controllable device 520. As another example, the user may usethe remote control 510 to drive remotely controllable device 520. Inembodiments, an action may occur, for example, when remotelycontrollable device moves, approaches another object, collides withanother object, an input and/or output device of the remotelycontrollable device is moved or fired. Numerous other embodiments aredisclosed herein and variations are within the scope of this disclosure.

In embodiments, when an action occurs in remotely controllable device520, remote control 510 sends information to computer 540 directlyand/or through network 530. For example, remote control 510 can sendinformation regarding the action to computer 540. Thus, in oneembodiment, as remotely controllable device 520 is driven or otherwiseoperated by a user using remote control 510, the remote control 510sends information regarding the movements to computer 540. Remotecontrol 510 can send sensor information received from remotelycontrollable device 520 to computer 540. For example, remote control 510may receive sensor information from remotely controllable device 520indicating that the remotely controllable device has collided withanother object and at least a portion of the sensor information may besent by remote control 510 to computer 540. In embodiments, computer 540receives the sensor information and/or action information from remotecontrol 510 through network 530. Numerous other embodiments aredisclosed herein and variations are within the scope of this disclosure.

Referring back to method 700, once the information has been receivedfrom the first remote control 710, the method proceeds to block 720. Inblock 720, an environment is updated. For example, referring to FIG. 5,if computer 540 receives information from remote control 510 throughnetwork 530, then computer 540 may update an environment based at leastin part on the received information. In one embodiment, the environmentis an augmented reality environment maintained by computer 540. Forexample, computer 540 may maintain an augmented reality by stitchingtogether or otherwise combining information receiving from variousremote controls, such as remote control 510 and remote control 515, tocreate a single environment. In embodiments, at least a portion of theinformation received from the various remote controls is overlaid withone or more graphical objects, videos, or other virtual environment tocreate a single, augmented environment. Thus, in an embodiment, asactions and/or sensor information from remote control 510 and/or remotecontrol 515 is received by computer 540, the augmented realityenvironment maintained by computer 540 is updated. For example, as auser interacts with remote control 510 to control absolute movements ofremotely controllable device 520, action information and/or sensorinformation can be sent to computer 540 and used by computer 540 toupdate the augmented reality environment by translating the absolutemovements into relative movements in the augmented reality environment.Numerous other embodiments are disclosed herein and variations arewithin the scope of this disclosure.

Referring back to method 700, once the environment has been updated 720,the method 700 proceeds to block 730. In block 730, information is sentto a second remote control. For example, referring to FIG. 5, computer540 may send information to remote control 515 through network 530. Inone embodiment, the information sent by computer 540 to remote control515 is sensor information and/or action information that the computer540 received from remote control 510. In another embodiment, theinformation sent by computer 540 to remote control 515 is informationusable to update a display associated with remote control 515 so that itreflects at least a portion of the updated augmented reality of thecomputer 540. For example, if computer 540 receives information fromremote control 510 indicating a wheel of remotely controllable device520 has come off, then computer 540 may update the augmented realityenvironment to indicate that the wheel of remotely controllable device520 has come off and computer 540 can send information to remote control515 can be used to update a display of remote control 515 to show a userthat remotely controllable device 520 no longer has a wheel.

As another example, if computer 540 receives information from remotecontrol 510 indicating that a gun or other weapon is disabled, thencomputer 540 may update the augmented reality environment to reflectthat the gun has been disabled and computer 540 may send statusinformation to remote control 515 indicating that the gun has beendisabled. In this embodiment, a display of remote control 515 may beupdated to reflect that the gun has been disabled. For example, an “X”may be overlaid over the gun that has been disabled and displayed on thedisplay. Numerous other embodiments are disclosed herein and variationsare within the scope of this disclosure.

General

While the methods and systems herein are described in terms of softwareexecuting on various machines, the methods and systems may also beimplemented as specifically-configured hardware, such asfield-programmable gate array (FPGA) specifically to execute the variousmethods. For example, embodiments can be implemented in digitalelectronic circuitry, or in computer hardware, firmware, software, or ina combination thereof. In one embodiment, a device may comprise aprocessor or processors. The processor comprises a computer-readablemedium, such as a random access memory (RAM) coupled to the processor.The processor executes computer-executable program instructions storedin memory, such as executing one or more computer programs for editingan image. Such processors may comprise a microprocessor, a digitalsignal processor (DSP), an application-specific integrated circuit(ASIC), field programmable gate arrays (FPGAs), and state machines. Suchprocessors may further comprise programmable electronic devices such asPLCs, programmable interrupt controllers (PICs), programmable logicdevices (PLDs), programmable read-only memories (PROMs), electronicallyprogrammable read-only memories (EPROMs or EEPROMs), or other similardevices.

Such processors may comprise, or may be in communication with, media,for example computer-readable media, that may store instructions that,when executed by the processor, can cause the processor to perform thesteps described herein as carried out, or assisted, by a processor.Embodiments of computer-readable media may comprise, but are not limitedto, an electronic, optical, magnetic, or other storage device capable ofproviding a processor, such as the processor in a web server, withcomputer-readable instructions. Other examples of media comprise, butare not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip,ROM, RAM, ASIC, configured processor, all optical media, all magnetictape or other magnetic media, or any other medium from which a computerprocessor can read. The processor, and the processing, described may bein one or more structures, and may be dispersed through one or morestructures. The processor may comprise code for carrying out one or moreof the methods (or parts of methods) described herein.

The foregoing description of some embodiments of the invention has beenpresented only for the purpose of illustration and description and isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed. Numerous modifications and adaptations thereof will beapparent to those skilled in the art without departing from the spiritand scope of the invention.

Reference herein to “one embodiment” or “an embodiment” means that aparticular feature, structure, operation, or other characteristicdescribed in connection with the embodiment may be included in at leastone implementation of the invention. The invention is not restricted tothe particular embodiments described as such. The appearance of thephrase “in one embodiment” or “in an embodiment” in various places inthe specification does not necessarily refer to the same embodiment. Anyparticular feature, structure, operation, or other characteristicdescribed in this specification in relation to “one embodiment” may becombined with other features, structures, operations, or othercharacteristics described in respect of any other embodiment.

That which is claimed is:
 1. A computer-readable medium comprisingprogram code for: receiving, by a portable multifunction device,information from a remotely controllable device, the portablemultifunction device being operable as a remote control for the remotelycontrollable device; determining, by the portable multifunction device,a haptic effect based at least in part on the received information;generating, by the portable multifunction device, a signal configured tocause an actuator to output the first haptic effect; and outputting, bythe first portable multifunction device, the signal.
 2. Thecomputer-readable medium of claim 1, wherein the portable multifunctiondevice is a smartphone.
 3. The computer-readable medium of claim 1,wherein the portable multifunction device is a watch.
 4. Thecomputer-readable medium of claim 1, wherein the remotely controllabledevice comprises at least one of a car, a plane, a drone, a robot, or adoll.
 5. The computer-readable medium of claim 1, wherein theinformation is received from the remotely controllable device over atleast radio frequency.
 6. The computer-readable medium of claim 1,wherein the information is received from the remotely controllabledevice via at least one of a cellular network or a Wi-Fi network.
 7. Thecomputer-readable medium of claim 1, wherein the information comprisessensor information associated with one or more sensors in the remotelycontrollable device.
 8. The computer-readable medium of claim 7, whereinthe sensor information comprises at least one of a direction, a tilt, ora speed of the remotely controllable device.
 9. The computer-readablemedium of claim 1, wherein the information comprises a request for theportable multifunction device to cede control of at least a portion ofthe remotely controllable device to a second portable multifunctiondevice.
 10. The computer-readable medium of claim 1, wherein theinformation comprises a video feed from at least one camera associatedwith the remotely controllable device.
 11. The computer-readable mediumof claim 10, further comprising program code for: overlaying at least aportion of the video feed on one or more graphical images; anddisplaying the overlaid view on a display of the portable multifunctiondevice.
 12. The computer-readable medium of claim 1, wherein thereceived information comprises environmental information for theremotely controllable device.
 13. The computer-readable medium of claim1, further comprising: receiving, by the portable multifunction device,information from a second portable multifunction device, the secondportable multifunction device being operable as a remote control for theremotely controllable device at the same time as the portablemultifunction device.
 14. The computer-readable medium of claim 13,wherein the information received from the remotely controllable devicecomprises environmental information for the remotely controllabledevice, wherein the information received from the second portablemultifunction device comprises an event performed by the second portablemultifunction device, and wherein determining the haptic effect based atleast in part on the received information comprises: determining thehaptic effect based at least in part on the environmental informationreceived from the remotely controllable device and the event receivedfrom the second portable multifunction device.
 15. The computer-readablemedium of claim 1, wherein the received information comprisesinformation usable to determine an event associated with the remotelycontrollable device.
 16. The computer-readable medium of claim 1,wherein the event occurs in an augmented reality.
 17. Thecomputer-readable medium of claim 1, further comprising: receiving, bythe portable multifunction device, information from a second portablemultifunction device, the second portable multifunction device beingoperable as a second remote control for a second remotely controllabledevice.
 18. The computer-readable medium of claim 17, wherein the firstportable multifunction device is in a first location, wherein the secondportable multifunction device is in a second location, wherein the firstlocation is different from the second location, and wherein the firstportable multifunction device in the first location is configured tocommunicate with the second portable multifunction in the secondlocation through at least one network.
 19. The computer-readable mediumof claim 17, wherein the information received from the second portablemultifunction device is in response to an event that occurred in anaugmented related during gameplay, and wherein the determined hapticeffect is based at least in part on the information received form thesecond portable multifunction device.
 20. A method comprising:receiving, by a portable multifunction device, first information from aremotely controllable device, the portable multifunction device beingoperable as a remote control for the remotely controllable device;determining, by the portable multifunction device, a first haptic effectbased at least in part on the received first information; generating, bythe portable multifunction device, a first signal configured to cause anactuator to output the first haptic effect; and outputting, by the firstportable multifunction device, the first signal.